Automatic Evaluation of Information Ordering: Kendall's Tau
نویسندگان
چکیده
منابع مشابه
Automatic Evaluation of Information Ordering: Kendall's Tau
This article considers the automatic evaluation of information ordering, a task underlying many text-based applications such as concept-to-text generation and multidocument summarization. We propose an evaluation method based on Kendall’s τ, a metric of rank correlation. The method is inexpensive, robust, and representation independent. We show that Kendall’s τ correlates reliably with human ra...
متن کاملAdaptive explicit-implicit tau-leaping method with automatic tau selection.
The existing tau-selection strategy, which was designed for explicit tau leaping, is here modified to apply to implicit tau leaping, allowing for longer steps when the system is stiff. Further, an adaptive strategy that identifies stiffness and automatically chooses between the explicit and the (new) implicit tau-selection methods to achieve better efficiency is proposed. Numerical testing demo...
متن کاملAutomatic language and information processing: rethinking evaluation
System evaluation has mattered since research on automatic language and information processing began. But the (D)ARPA conferences have raised the stakes substantially in requiring and delivering systematic evaluations and in sustaining these through long term programmes; and it has been claimed that this has both significantly raised task performance, as defined by appropriate effectiveness mea...
متن کاملOrdering as privileged information
We propose to accelerate the rate of convergence of the pattern recognition task by directly minimizing the variance diameters of certain hypothesis spaces, which are critical quantities in fast-convergence results. We show that the variance diameters can be controlled by dividing hypothesis spaces into metric balls based on a new order metric. This order metric can be minimized as an ordinal r...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Computational Linguistics
سال: 2006
ISSN: 0891-2017,1530-9312
DOI: 10.1162/coli.2006.32.4.471